Method and apparatus to layout screens of varying sizes

ABSTRACT

Different methods and apparatuses applicable to prepare materials for a user. One embodiment includes a device, with an imaging sensor. Based on analyzing a user attribute from the imaging sensor monitoring the user, the device can identify and access an area of the materials for the user. The area includes a section, and the device can layout the section by keeping a piece of text together with an illustration to be displayed in at least two screens of different sizes. Another embodiment includes materials with a text sub file with texts, an illustration sub file with illustrations, and a logic sub file with instructions to prepare materials for the user. Either the text or the illustration sub file can include position information linking at least an illustration to a corresponding piece of text. Annotations could be added to the materials.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims benefit of

-   -   (I) U.S. patent application Ser. No. 17/160,714, filed Jan. 28,        2021, and entitled “Method and Apparatus to Layout Screens of        Varying Sizes,” which is hereby incorporated herein by        reference;    -   (II) U.S. patent application Ser. No. 16/692,101, filed Nov. 22,        2019, now U.S. Pat. No. 11,074,393, and entitled “Method and        Apparatus to Layout Screens,” which is hereby incorporated        herein by reference;    -   (III) U.S. patent application Ser. No. 15/079,911, filed Mar.        24, 2016, now U.S. Pat. No. 10,503,812, and entitled “Method and        Apparatus for Materials in Different Screen Sizes Using an        Imaging Sensor,” which is hereby incorporated herein by        reference;    -   (IV) U.S. patent application Ser. No. 12/804,875, filed Jul. 30,        2010, now U.S. Pat. No. 9,330,069, and entitled “Layout of        e-Book Content in Screens of Varying Sizes,” which is hereby        incorporated herein by reference;    -   (V) U.S. Provisional Patent Application No. 61/278,952, filed        Oct. 14, 2009, and entitled “A Computer-aided dynamic reading        and learning method and system,” which is hereby incorporated        herein by reference;    -   (VI) U.S. Provisional Patent Application No. 61/281,063, filed        Nov. 12, 2009, and entitled “Computer-aided dynamic reading and        learning method and system,” which is hereby incorporated herein        by reference;    -   (VII) U.S. Provisional Patent Application No. 61/283,687, filed        Dec. 8, 2009, and entitled “Computer-aided reading and learning        methods and systems,” which is hereby incorporated herein by        reference;    -   (VIII) U.S. Provisional Patent Application No. 61/336,156, filed        Jan. 16, 2010, and entitled “Computer-aided reading and learning        methods and systems,” which is hereby incorporated herein by        reference; and    -   (IX) U.S. Provisional Patent Application No. 61/338,991, filed        Feb. 27, 2010, and entitled “Computer-aided reading and learning        methods and systems,” which is hereby incorporated herein by        reference.

BACKGROUND OF THE INVENTION Description of the Related Art

A number of companies have been introducing e-reading devices into themarket. Typically books sold via such devices are cheaper than theircorresponding paper versions. Also, such devices allow readers anytimeaccess to huge number of books. They do not have to physically carry thebooks. Instead, the devices have the capacity to store lots of books,and can wirelessly download additional ones, even while the readers areon a train or waiting for an appointment. Publishers should be worriedthat the sales of their printed books would go down. However, afterusing the devices, many people are finding they miss the feel and touchof the paper books. Though such devices appear to have numerousadvantages, the market is not experiencing a wide adoption of suchdevices to read books.

It should be apparent from the foregoing that there is still a need fora better reading and learning method and system for e-books.

SUMMARY OF THE INVENTION

In one embodiment, the present invention can provide an e-book readingdevices for e-books with illustrations tied to their correspondingtexts, with pagination preserved, with bookmarks available, and withannotations dynamically tailored to a reader and appropriatelypositioned for the reader, where notes from the reader can be easilycaptured and shared. The e-books can be children books, text books,cookbooks, journals, newspaper and many other reading materials orconsolidations of different reading materials.

The invention can be implemented in numerous ways including, a method,system, device, and a computer readable medium. Other embodiments of theinvention are discussed below.

One embodiment of a portable e-book reading device can dynamicallypaginate, linking and presenting text and illustration in pages on ascreen of a display in a user-friendly way. Each page can be shown onone or more consecutive screens of the display, with the paginationmaintained. The display can include a LCD display. In one embodiment, anillustration can be an animation or a video. In another embodiment, anillustration can be an audio file.

Another embodiment of a portable e-book reading device is configured forreading a paginated e-book, where each page includes one or moresections, each section includes an expression, and at least one sectionincludes a piece of text linked to an illustration. The device includesa controller, a display configured to show a number of screens, an inputmechanism to allow a reader to provide input into the device, and ascreen layout module coupled to the controller and the display. Thescreen layout module can layout a number of pages of the e-book. Inlaying out the piece of text linked to the illustration, the screenlayout module can at least keep the piece of text with the illustrationto be displayed in one screen, and prevent the illustration from beingsplit apart to be displayed in more than one screen. In addition, thescreen layout module can maintain the pagination if a page is displayedin more than one consecutive screen. The display can be a LCD display.In one approach, the device can have more than one display. For example,one display can be an e ink display and the other a LCD display; onedisplay can be for text and the other for illustration; and/or one forreading and the other for the reader to annotate.

In one embodiment, a reading device includes a note selector that canretrieve an annotation with position information regarding theannotation for an e-book. And a screen layout module can assemble theannotation to a position (based on the position information) in thee-book for display. The note selector can include an annotation ofanother person, such as from an e-book club. In one approach, a piece oftext in an e-book is related to an annotation. If the reader highlightsthe piece of text, the annotation can be displayed.

One embodiment of a reading device includes a note editor that canassemble an annotation entered by a reader through an input mechanism ata reading device. The annotation can be linked to a section of thee-book, which can be specified by the reader. The note editor can keeptrack of the identity of the person creating the annotation.

In one embodiment, an e-book includes a text sub file that can be usedto store texts, together with corresponding attributes regarding thetexts, such as format and position information of the texts; anillustration sub file that can be used to store illustrations, withposition information linking the different illustrations to theircorresponding texts, if any, in the text sub file; and a logic sub fileincluding at least one rule to instruct a screen layout module on howsections of the e-book should be displayed on a number of screens of adisplay. In one embodiment, the text sub file can be embedded in thelogic sub file. In another embodiment, the e-book can include anauxiliary sub file with at least an annotation and information regardingthe annotation, such as position information as to where the annotationis to be positioned in the e-book. In one approach, the annotation canbe a blog. There can be an annotation control mechanism to control if anannotation is to be displayed on a screen.

In one embodiment, an e-book can be provided via a website, which canprovide a catalogue of annotations for a reader to search forannotations to be added into the e-book. The searched results, such asthe priority of the searched results, can depend on an interest of thereader, such as the content in the e-book that the reader has recentlybeen reading. The reader can decide the position in the e-book where anannotation should be linked.

In one embodiment, an annotation can be an audio file. In anotherembodiment, an illustration can be an audio file. The audio file can belinked to a section of an e-book. The playing of the audio file can beautomated, or controlled by a reader.

One embodiment of a reading device includes a reader monitor that canmonitor a reader, such as the reader's reading behavior. An auxiliarysub file in an e-book can record at least one feature monitored by thereader monitor. Examples of information being recorded can include thereader's inputs in controlling page movements; the frequency the readervisited a section or a page of the e-book; the duration of time thereader spent at a section or a page of the e-book; the reader'sannotation, with a corresponding position of the annotation in thee-book; a page dog-eared by the reader, which, if activated, can bringthe page onto the screen; a section dog-eared by the reader, which, ifactivated, can highlight the section on the screen; and/or the reader'simage, which can be recorded, and which can be linked with thecorresponding page or section of the e-book when the image was taken.

In one embodiment, a screen layout module includes a screen layoutengine to assemble contents in a screen, and a screen rendering engineto digitize the assembled contents. The screen layout engine can receiveinformation from an illustration descriptor in an illustration sub file,and information from a text sub file to assemble the contents in thescreen. The screen rendering engine can digitize text from the text subfile and illustrations from the illustration sub file for display. Inone embodiment, the screen rendering engine can also scale anillustration in view of available space in a screen. As to the timing ofthe layout process, the screen layout module can assemble the contentsin a screen for a later part of the e-book as the reader is reading anearlier part of the e-book. In another embodiment, the e-book can befrom a web site, and the web site can assemble at least a portion of thecontents in the e-book to be displayed, and transmit the assembledcontents to the reading device as the e-book is downloaded from the website.

In one embodiment, to layout contents for a page with text, a screenlayout module can first determine the size of a screen of the displayand can retrieve a predetermined font size for the text. Then, themodule can retrieve at least a section of the page to be displayed, andcalculate the logical position of the section in the screen. Based onthe calculation, the module can determine if the current screen hassufficient space to display the section. If there is sufficient space,the module can display the section of text. If there is insufficientspace, the module can move the section to the next screen.

In one embodiment, a text sub file includes both information to specifythe layout of the text and the corresponding illustrations. To layoutsuch a page, a screen layout module can retrieve such information fromthe text sub file to perform the assembling. In another embodiment, anillustration sub file includes both information to specify the layout ofillustrations and the corresponding text. To layout such a page, ascreen layout module can retrieve such information from the illustrationsub file to perform the assembling.

In one embodiment, a page includes both text and illustration. As thereader is reading the text, a symbol can alert the reader of acorresponding illustration. If the reader selects or activates thesymbol, the illustration can be displayed. In another embodiment, as thereader is reading or watching an illustration, a symbol can alert thereader of a piece of text. If the reader selects the symbol, the textcan be displayed.

In one embodiment, a first part of the screen can be reserved for text,and a second part of the screen can be reserved for illustration. If apage has both text and illustration, the text can be displayed at thefirst part of the screen and the illustration at the second part. In oneembodiment, if the illustration is larger (e.g. has more bits) than theavailable space in the second part, a screen layout module can scale theillustration to fit into the second part of the screen. In oneembodiment, if the scaling exceeds a pre-determined percentage, thescreen layout module can move the illustration with the correspondingtext, if any, to the next screen.

In another embodiment, a screen layout module layouts contents in ascreen for a page with text and illustration. First the module candetermine the size of the screen, and reserve a first portion of thescreen for text, and a second portion for illustration. Then the modulecan retrieve a preset font size for the text. The module can retrieve atleast a section of the page to be displayed, and calculate the logicalposition of the section in the screen. Based on the calculation, themodule can determine if the first section of the screen has sufficientspace to display the section. If there is insufficient space, the modulecan move the piece of text with all subsequent sections of the page tothe next screen. If there is sufficient space, the module determines ifthere is a corresponding illustration linked to the text. If there isnot, the text would be displayed. If there is, the module can thenretrieve at least one attribute regarding the corresponding illustrationto be displayed, and calculate at least one logical position for theillustration. Based on the calculation, the module can determine if thesecond section of the screen has sufficient space to display theillustration. If there is sufficient space, the module can retrieve theillustration and display both the illustration with the correspondingpiece of text on the screen. If there is insufficient space, the modulecan move the piece of text with the corresponding illustration to thenext screen.

In one embodiment, to display an annotation for a section, a screenlayout module can search for the annotation corresponding to the sectionamong a number of annotations in an auxiliary sub file. If such anannotation exists, the annotation is identified, and the module canretrieve the annotation to be displayed with the corresponding section.In another embodiment, the screen layout module again can determine ifthere is sufficient space in the current screen to display theannotation. If not, the annotation with the corresponding section can bemoved to the next screen.

In one embodiment, a reader analyzer can analyze information captured bya reader monitor to determine at least one attribute of the reader. Thereader analyzer can determine, for example, a reading pace of the readerin reading an e-book; the reader's degree of interest in at least asection of the e-book (which can be determined based on, for example, areading behavior of the reader or whether the reader is in front of thereading device); the reader's concentration level in an area of thee-book; and the reader's understanding level in an area of the e-book(which can be based on a response by the reader to a question from thedevice). In determining an attribute, in one approach, the analyzer cancompare an attribute of the reader to an average measurement of theattribute of a number of other readers. In another approach, indetermining an attribute, the analyzer can identify the subject matterof the area the reader is reading.

In one embodiment, a reader responder can respond to the reader based onat least one attribute of the reader. The responder can ask the readerif the reader wants additional materials related to an area of thee-book. The additional materials can previously be locked in the readingdevice, and can be unlocked to allow the reader to read. In oneembodiment, the reader needs to be a member of an e-book club to readthe additional materials. In one approach, the responder can present acatalogue of titles to the reader to select. Upon selection, theresponder can retrieve the additional materials under the selected titlefrom such as a web site for the reader. In one embodiment, theadditional materials can be in a memory device of another person, andthe reading device can wirelessly and/or via wire to access the memorydevice. In different embodiments, the responder can provide reviewsregarding certain materials in an e-book for the reader (the reviews canbe related to but different from the certain materials); reward thereader when the reader reaches a pre-determined milestone (the rewardcan be tailored to an interest of the reader); restrict the reader froman activity under a pre-determined condition; and/or guide to the readera product or a service the reader is interested in.

In one embodiment, to read an e-book, the reader needs to enter anappropriate identification into the e-book, and a different e-book canrequire a different identification.

In one embodiment, different components or functionalities of a readingdevice to read an e-book can be in the e-book. For example, at least aportion of a reader analyzer is in the e-book. In another example, atleast a portion of a reader analyzer and at least a portion of a readerresponder can be in an e-book; such as the portion of the analyzer inthe e-book categorizes the reader into one of many categories, and inview of the categorizing, the portion of the responder in the e-bookretrieves at least a different section of the e-book to be presented tothe reader. In another example, the responder retrieves the meaning ofat least one word from a dictionary based on the categorizing, and thedictionary can be in the e-book. For example, readers belonging to afirst age group get an image as the meaning for the at least one word,and readers belonging to a second age group gets a piece of text as themeaning of the at least one word. In one embodiment, the portion of thereader analyzer and the portion of the reader responder in an e-book canbe downloaded from the e-book into a reading device to be used by thereading device. In one embodiment, at least a portion of a screen layoutmodule is in an e-book. In one embodiment, a functionality to read asection of an e-book can be downloaded from the e-book into a readingdevice when a reader is reading the e-book, but before the readerreaches the section.

One embodiment includes an e-book for a reader with a text sub file anda logic sub file. The text sub file can be configured to store at leastone piece of text, together with at least an attribute regarding the atleast one piece of text. The logic sub file can be configured to includea set of instructions to instruct an e-book reading device on afunctionality regarding reading the e-book at the device. Thefunctionality can include analyzing an attribute of the reader regardingreading the e-book, and providing a response to the reader via thedevice in view of the attribute analyzed. In one embodiment, the e-bookfurther includes an illustration sub file to store at least anillustration, together with at least an attribute regarding theillustration. Either the text sub file or the illustration sub file caninclude a piece of position information linking the illustration to anarea at the at least one piece of text in the text sub file.

In one embodiment, an e-book reading device includes a speaker, and thee-book can be read to a reader via the speaker. In one approach, pagenumbers can be read by the speaker as the speaker is reading the e-book.

Other aspects and advantages of the present invention will becomeapparent from the following detailed description, which, when taken inconjunction with the accompanying drawings, illustrates by way ofexample the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows one embodiment according to the invention of a screenshowing text and animation from an e-book.

FIG. 2 shows one embodiment according to the invention of a screen withtext, animation and annotations.

FIG. 3 shows one embodiment according to the invention of a screen withtext and animation, with annotations from a club.

FIG. 4 shows one embodiment of a reading device according to theinvention.

FIG. 5 shows one embodiment with a number of components in a readingdevice according to the invention.

FIG. 6 shows one embodiment according to the invention of a number ofsub files of an e-book.

FIG. 7 shows a number of attributes regarding a reader monitor accordingto different embodiments of the invention.

FIGS. 8A-C show different embodiments of a text sub file and anillustration sub file according to the invention.

FIG. 9 shows one embodiment according to the invention of a screenlayout module with different inputs.

FIG. 10 shows different embodiments of a screen layout module to layoutcontents in different pages for an e-book according to the invention.

FIG. 11 shows one embodiment of an annotation descriptor based on XMLaccording to the invention.

FIG. 12 shows one embodiment of a collection of annotations based on XMLaccording to the invention.

FIG. 13 shows a number attributes regarding a reader analyzer accordingto different embodiments of the invention.

FIG. 14 shows a number of embodiments according to the inventionregarding attributes of a reader responder.

FIG. 15 shows features in a logic sub file of an e-book according todifferent embodiments of the invention.

FIGS. 16A-B show examples of hardware to implement one or moreembodiments according to the invention.

Same numerals in FIGS. 1-16 are assigned to similar elements in all thefigures. Embodiments of the invention are discussed below with referenceto FIGS. 1-16. However, those skilled in the art will readily appreciatethat the detailed description given herein with respect to these figuresis for explanatory purposes as the invention extends beyond theselimited embodiments.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 shows one embodiment of a screen of a visual output mechanism(like a display) of an e-book reading device. The screen shows anexpression from an e-book. In this embodiment, expressions in differentforms can be linked together on a number of pages in a user-friendlyway. An expression can be a piece of text, an illustration, an audioclip, or others. In one embodiment, an illustration generally describesan expression that can dynamically change when displayed. In oneembodiment, an illustration can be an animation or a video. An animationor a video can show, for example, a continuous action, a flow of ideas,an experiment, a reconstruction of history, a visualization of theory,and/or a made-believe moment. In another embodiment, an illustration canbe an audio file, such as an mp3 file. In one embodiment, an e-book canbe a consolidated set of expressions that can standalone and can bepresented without additional expressions for a reader's readingenjoyment.

In one embodiment, a page can have one or more sections. Each sectionincludes an expression. The expression can be, for example, a word, aline of the text, multiple lines of the text, and/or an illustration. Asanother example, a section can include a piece of text linked to anillustration. In the embodiment shown in FIG. 1, a page has a pagenumber to identify the page, and the page number in FIG. 1 is 35. Suchpagination determination can be set, for example, by the author orcreator of the e-book. In another example, a publisher of an e-bookdetermines how the e-book should be paginated and the content in eachpage. Each page can be shown on one or more consecutive screens of asystem. Though a page can be shown in multiple screens, the paginationof the page can remain the same. In one embodiment, a screen shows onlyone page or part of one page, and does not show more than one pagesimultaneously. The illustration can be shown on a screen, but not parton one screen, and part on another screen, such as the top part of anillustration on one screen, and the bottom half of the illustration onthe following screen. Also, in one embodiment, each illustration isshown with (or linked to) its related text or the corresponding sectionwith the text, if any, on the same screen.

In FIG. 1, a piece of text, or a section with the text, is linked to ananimation, which is related to and which reinforces the content of thetext, or the section with the text. In the figure, one image of theanimation is shown. Also, in the figure, a number of words arehighlighted, such as Grievous and lightsabers. For additionalexplanation, these highlighted words can be selected, which would thenhypertext-link to other areas. The hypertext linkage of a word can be toa dictionary for the meaning of the word, to an Internet search websitefor more information about the word, or to other areas.

In one embodiment, as a reader reads an e-book, additional notes orannotations can be presented to the reader regarding the area where thereader is reading. Such notes can help the reader learn about a subject.In one example, these notes are shown alongside their related textand/or illustration. In one embodiment, an e-book can be referred to asthe e-book originally published by a publisher, which, in one approach,can be the author or creator of the e-book; and an expressionsubsequently added to the e-book can be referred to as an annotation ofthe e-book. In another embodiment, an annotation can be an expressionadded to an e-book, can be in the e-book's 1st or original publication,and the e-book can standalone without the annotation for a reader'sreading enjoyment. In yet another embodiment, the expressions in ane-book can be written or created by one or more than one person, whilethe annotation can be generated by a separate person.

FIG. 2 shows one embodiment of a screen with text, animation (with justone picture of the animation shown in the figure) and annotations for ane-book. In one approach, the text and animation can be from the e-bookoriginally published by a publisher of the e-book for a reader to read.The annotations can be from another reader (or another person affiliatedto the e-book) of the e-book, such as a Star War fan in this example. Apiece of note can be more widely accepted/read if it is from a respectedperson in the field, such as a classroom teacher teaching thecorresponding subject, a good critic in the field or a recognizedauthor. These people are typically perceived to be more credible orknowledgeable particularly in the area indicated in the note. In FIG. 2,one such person can be Old-Grievous. In one embodiment, a piece of textwith a related annotation can be highlighted, and selecting thehighlighted piece of text leads to the display of the relatedannotation. For example, the piece of text can be hypertext linked to anannotation of a person. By selecting the piece of hypertext-linked text,the annotation can show up on the screen to be read.

In another embodiment, the reader can also write notes or annotate,which can create a more interactive way of reading. Many readers enjoysharing their reading experiences and/or insights. Expressing theiropinion not only can enhance their joy of reading, but can alsoreinforce their understanding of the material. Exchanging such notes orideas often goes beyond reinforcing understanding in the readingmaterials. Such active exchange of information can create a morepermanent imprint of the read materials.

FIG. 3 shows an embodiment where the notes can be from a fan club. As areader is reading an e-book, notes from a club (which can be at awebsite) can also be included into the e-book for the reader to read. Inthe example shown in FIG. 3, the fan club is the Grievous Fan Club.

FIG. 4 shows an embodiment of a reading device or system. In thisembodiment, the system can be a portable handheld device with a LCDdisplay. The device includes a wireless connection port configured toaccept different wireless connection modules for different wirelessconnection formats, such as WiFi, WiMax and cellular connection. Thedevice also can include, among other features, a memory card slot, and aPC connection slot. The memory card slot is configured to accept, forexample, different memory cards, a USB drive, or a detachable storagemedium. The PC connection slot is configured, for example, to connectthe device to another computing device. A page can be shown on thedisplay of the reading device on FIG. 4. In another embodiment,different wireless connection mechanisms or modules are included in thereading device.

In the embodiment shown on FIG. 4, the device also includes a next pageand a next screen button, which can be software buttons or physicalbuttons on the device. The next-page button allows a reader to read thenext page, while the next-screen button allows the reader to flip to thenext screen, which can be the same page.

Another embodiment allows reading an e-book via a device similar to acellular phone. The phone can have a screen that is about 2 inches wideand 3.5 inches tall, such as having the size of a smart phone screen. Inanother example, an embodiment has a screen about the size of apaper-back novel. In yet another embodiment, its screen can be about 8.5inches by 11 inches.

In one embodiment, to provide inputs to the device, the device includesan input mechanism, such as a keypad, a writing pad, or keyboard, whichcan be a physical or a software version. The inputs provided can beinformation, such as annotations by a reader. Another embodimentincludes a touch sensitive screen, which can be a resistive touchscreen. In another example, the touch sensitive screen is a capacitivetouch screen, which can be more applicable to finger than to stylusmanipulation. In one approach, the touch sensitive screen can be anoverlay over a LCD display.

In one embodiment, an illustration can start if a readeractivates/clicks it on the screen. The illustration can be paused by thereader, such as the reader pushing a pause button, which is not shown onthe device in FIG. 4.

One embodiment allows a reader to open a window at, for example, thebottom of the screen, to provide inputs, such as notes, into the device.Notes can be entered via a keyboard or a pen stylus directly into thewindow. In one approach, the device can include character recognitionsoftware to interpret the inputs. Yet another approach includesprediction software to predict what the reader is writing based on whatthe reader has entered. Such character recognition and predictionsoftware should be known to those skilled in the art and would not befurther described in this application.

Another embodiment includes more than one display in the reading device.In the case of a dual-display version, the two displays can bepositioned side by side. In one embodiment, one display can be based one ink technology, while the other can be a LCD display. The e inkdisplay can be used to display text, while the LCD display can be usedto display illustration, such as video. In another embodiment, onedisplay can be for reading, while the other can be for enteringannotation.

FIG. 5 shows one embodiment of a reading device with a number ofcomponents. The device can include one or more of the followingcomponents coupled together: a controller (such as a processor), ascreen layout module, a display configured to show a number of screens,a note selector, a note editor, a reader monitor, a reader analyzer anda reader responder. In one embodiment, the screen layout module isconfigured to control the layout of the content of an e-book (or asection of an e-book) in each screen. In another embodiment, the noteselector is configured to retrieve or select an annotation with aposition in the e-book. As an example, in FIG. 5, the note selector hasretrieved study guide 1, study guide 2, note 1 and note 2 for a reader.After the notes have been retrieved, the screen layout module can beconfigured to assemble the notes into the corresponding positions in thee-book for display on a screen. In another embodiment, the readermonitor can be configured to monitor the reader, such as the reader'sreading behavior; and the reader analyzer is configured to analyze, forexample, the interest and/or progress of the reader based on informationregarding the reader, such as the information captured by the readermonitor. Then based on the results from the analyzer, the readerresponder can be configured to respond accordingly. For example, thereader responder can instruct the note selector to select a piece ofnote for the reader based on the analyzer-identified reader's interest.In one embodiment, the reader responder can be configured to respond tothe reader based on the reader's inputs, as indicated by the readermonitor. For example, a reader requests connection to the Internet atthe e-book reading device, and the responder can connect the e-book tothe Internet to allow reader access.

In another embodiment, the reading device includes a note editor, whichcan be configured to assemble an annotation entered through an inputmechanism. For example, a reader enters a piece of note. The readermonitor can capture the piece of note entered together with the sectionwhere the note is entered or where the note is designated to be linked.For example, the reader can designate the position where the note shouldbe linked to. Then the note editor can link the piece of note to thecorresponding area or section in the e-book, and can have the piece ofnote stored with the corresponding pagination, area and/or sectioninformation. The note editor can also add a symbol or bookmark, such asa carrot, at the area or section in the e-book. Or, the editor canhighlight one or more words in a piece of text, or an area in anillustration, where there is an annotation. Subsequently, one way tobring up the annotation can be by selecting the bookmark or thehighlighted area. In another embodiment, the note editor also keepstrack of the identity of the author or creator of the piece of note, andlinks the identity to the piece of note.

FIG. 6 shows one embodiment using a number of sub files, for example, tohold different files/aspects of an e-book (or a portion of an e-book).In one implementation, there are 3 sub files, including a text sub file,an illustration sub file, and a logic sub file. The text sub file can beconfigured to hold the text portion of an e-book, together with at leastone attribute regarding the text, such as the format and positioninformation of words in the text. In one example, the text sub file canbe based on HTML and/or XML format, or other text marked-up format. Theillustration sub file can be configured to hold the illustration portionof the e-book. In one embodiment, the illustration sub file includes anillustration descriptor and the illustrations. An illustrationdescriptor includes at least one attribute regarding an illustration,such as information linking the illustration to its corresponding textsor section of text, if any, in the text sub file. The logic sub file canbe configured, for example, to include the logic (or at least one rule)to instruct a screen layout module on how, for example, a piece of note,illustration, text, paragraph, section or page of the e-book should bedisplayed on a screen. The logic sub file can be based on scriptinglanguage in HTML and/or XML format, such as Javascript, Actionscript,Silverlight script, and HTML script. In one implementation, a text subfile is embedded in a logic sub file.

One embodiment further includes an auxiliary sub file for the e-book. Inone approach, the auxiliary sub file includes an annotation, such asintroduced from other sources, for a reader, and the appropriatepagination, section and/or position information, where the annotationshould be positioned in an e-book. Then, for example, a screen layoutmodule can add the annotation into the appropriate page, area and/orsection of the corresponding text in a text sub file and/or thecorresponding illustration in an illustration sub file of the e-book. Inone embodiment, when a publisher publishes an e-book, the e-book doesnot have to include the annotation. For example, the annotation can beone or more text sub files and/or illustration sub files from othersources, such as other e-books. As another example, if the annotation isa blog, to locate correctly, the blog includes position information toidentify the position(s) of the corresponding page/section of the e-bookwhere the blog or different parts of the blog would be linked.

In one embodiment, an annotation introduced does not have to bedisplayed. For example, the logic sub file includes control mechanismsto activate viewing of the annotation. In one approach, annotations areat a number of positions in the e-book, and the activation mechanismscan be at each of the positions, such as at the correspondingpage/section. If activated at a certain page/section, the correspondingannotation can be displayed for that certain page/section of the e-book.In another embodiment, the activation mechanism is global for thee-book. If activated, annotations can be displayed whenever they occuracross the e-book. In yet another embodiment, activation can be based oncertain characteristics, such as all annotations by a certain person,but not another person, would be activated or automatically displayed attheir corresponding positions.

In one embodiment, a reader can be provided with a catalogue ofannotations. The annotations can be grouped together based on differentrules. For example, the annotations can be grouped together according totheir authors or creators. In another example, the annotations can begrouped together according to their subject matter. In yet anotherexample, the catalogue can be provided to the reader by, for example, aweb site that provides the e-book. The catalogue can include texts,illustrations or other expressions for the reader. The reader canelectronically search the catalogue and select the expressions that thereader wants. Such selected expressions can then be incorporated oradded to the e-book as annotations or additional materials for thereader. The incorporation can be through adding the expressions into thee-book's auxiliary sub file. As the reader reads the e-book, the addedexpressions can be brought in at the specific pages/sections asspecified by, for example, pagination/position information in the addedexpressions. In another approach, the reader can decide which section(s)in the e-book one or more added expressions should be linked to.

In one embodiment, after the reader has accessed an e-book, the readercan, at a later time, ask for additional expressions/annotations. In oneembodiment, the reader can ask for a catalogue of annotations (oradditional materials), and then electronically search for theannotations to be added into the e-book. One way is to go to the samesource, such as an e-book club or a web site, that provides the e-bookfor such annotations.

In one approach, searched results for the annotations can be based oncontexts of the e-book. For example, if the approximate position wherethe reader most recently is reading at the e-book is known, searchedresults can depend on the content/context at that position. Thecontent/context can help narrow the searched results and can providemore relevant search results for the reader. For example, more relevantsearched results can be ranked higher and listed earlier for the readerto select.

In one embodiment, an annotation can be an audio file. For example, whena page is read, an audio file (e.g. an mp3 annotation) is played. In oneapproach, the playing of the audio file can be automated. As the readermoves to a certain page, or a certain section, an audio file can beautomatically played. In another approach, a reader is alerted to anaudio file next to a section of an e-book. The reader can open oractivate the audio file and listen to it as the reader is reading thee-book.

FIG. 7 shows different embodiments regarding attributes of a readermonitor, which can be used to monitor a reader's reading behavior. Inone embodiment, the monitored information or attributes can be recordedin an auxiliary sub file. The reader monitor can monitor, for example,the reader's inputs in controlling page movements, such as whichpage/section the reader has moved by (or read) and the pace (such aspages/sections per minute); which page/section the reader has skipped(or moved by quickly relative to the reader's typically reading pace inreading the e-book, such as at more than 3 times of the typical pace);the frequency the reader has visited a section/page; the frequency thereader has watched an illustration; the frequency of a certain sectionof an illustration the reader has watched; the duration of time thereader has spent reading and/or watching a section/page; the number oftimes the reader has gone back to a section/page; and/or the number oftimes an animation has been replayed.

Another type of reading behavior can include a reader's annotationentered via, for example, an input mechanism at a reading device. Anannotation can be an expression, such as a piece of text entered by thereader; a picture, which can be from another source; or an illustration,which again can be created or retrieved from another source. Anannotation can also be a consolidation of different sets or types ofexpressions, such as texts and pictures from multiple e-books, sectionsin multiple e-books, blogs, newspapers and/or magazines. An auxiliarysub file can keep track of the corresponding section/page of the e-bookwhere the reader's annotation is related to or is entered at. Thetracking or position identification can be through bookmarking or by asymbol. For example, the reader can identify an area on a page and startannotating. The added notes can be bookmarked to the area, or denoted bya symbol at that area. Subsequently, clicking the bookmark or the symbolwould bring up the added notes, which can include the materialsintroduced by the reader. In one approach, the added notes can be showninside a window or a bubble, with a pointer pointing at thecorresponding section or area of the e-book.

In one embodiment, another type of reading behavior monitored by areader monitor can include recording one or more pages (or sections)dog-eared by a reader, such as by having a tab shown on a side of apage. The position of the tab relative to its corresponding page can bechanged. One approach includes a tab function, which can be a softwareor hardware switch. If the tab function is activated, the one or moretabs can be shown on the screen, such as a list of tabs shown on theright side (or another side, like the left side) of a page, or a visualrepresentation of multiple pages. In one example, a page is selected andis shown on a screen if its corresponding tab is activated, such asclicked or selected.

In one embodiment, a tab not only can identify a page, but the tab canalso identify a specific portion of the page, such as a section. Forexample, the tab can identify, such as, a specific paragraph, or anillustration in the e-book. When the tab is selected, the correspondingsection can be identified and highlighted. For example, when the tab isselected, the corresponding section can be brought to the center of thescreen.

Yet another type of reader's behavior can include the reader's images.In one embodiment, a reading device includes a camera (which can beconsidered as an input mechanism) that can be used by a reader monitorto capture the reader's images as the reader is reading. The images caninclude facial images. For example, when the camera takes a picture andthe reader is not in front of the camera, the picture would not includethe reader. Then the picture can be processed through, for example,pattern recognition techniques known to those skilled in the art toindicate that the reader is not (or is) in front of the device.Additional information can also be recorded. For example, page/sectioninformation can be recorded to indicate the associated page/section whena picture was taken.

Time information can be also recorded. For example, the time when animage is taken can be recorded. Such information can be linked with theassociated page/section. In another example, the duration of time when ascreen has been shown before another screen is displayed, can also berecorded.

A number of sub files of an e-book have been described. There aredifferent ways to implement the different sub files. FIG. 8A shows anembodiment of a text sub file. This embodiment shows certain text in apage with page number 35. The text also shows, for example, the positionand identification (ani-4) of an illustration with the name of Grievous.FIG. 8B shows an embodiment of an illustration descriptor of anillustration sub file. The illustration descriptor describes attributesof an illustration named ani-4, including the type of the illustrationbeing an animation; the dimensions of the animation; and the link“grevious.swf” for the animation. This link can be a link to theInternet. The different embodiments can be based on HTML and/or XML.

In one embodiment, an illustration can be an audio file. FIG. 8C showsan illustration descriptor that can be used for an audio file. In thisexample, the link of “starwars-track-04.mp3” shows an mp3 audio file.The attributes indicate that the audio file can play automatically whenits corresponding screen or section is displayed or rendered, with themusic file repeating until a user selects another screen or section. Inanother example, a music file can be linked to a piece of text (or anicon or symbol) and when a reader selects or activates the piece of text(or the symbol), the music can be played. In the example shown in FIG.8C, the dimension is set to be 0 pixel horizontally and vertically,indicating the illustration does not take up any display area of thescreen. In another example, an audio illustration can include a picture,which occupies some display area. There can be one or more otherattributes for audio files, such as, an audio illustration including aplaylist of music files; the corresponding illustration descriptorincluding the duration or the length of time of different pieces ofmusic; and the corresponding illustration descriptor including a presetvolume or other acoustic effects regarding the music. In one embodiment,a reading device provides physical or logical switches or controlmechanisms to allow a reader to control such acoustic effects.

In one embodiment, a note editor can be configured to generate a textsub file and/or an illustration sub file based on the annotationsentered and/or retrieved by a reader. For example, a reader enters apiece of text and retrieves a video as notes for a section in an e-book.The note editor can create a text sub file to store the text entered.The note editor can also create an illustration sub file with anillustration descriptor to denote attributes of the video, and with anillustration, which includes the content of the video.

In one embodiment, a screen layout module can be configured to layoutthe contents in one or more pages of an e-book (or one or more sectionsof an e-book), which can have different types of contents, for eachscreen. As an example, a page can have, for example, sections of text,each with its related illustration, with the first animation at the endof first paragraph, and the first video at the end of the third sentenceof the third paragraph in the page. Then the screen layout module canassemble the contents so that the first animation, the first videotogether with the text from the first paragraph up to the end of thethird sentence of the third paragraph are all displayed on the samescreen.

In one embodiment, a screen layout module includes a screen layoutengine and a screen rendering engine. For example, the screen layoutengine can be configured to assemble contents in each screen. The screenlayout engine can receive information from an illustration descriptor inan illustration sub file, and information from a text sub file.Information from the illustration descriptor can provide differentlayout attributes regarding a corresponding illustration (such as thedimensions of an animation). Information from the text sub file canprovide text and layout attributes regarding the text (such as font sizeof the text and the positions of different sections of the text). Basedon such layout information, in one embodiment, the screen layout enginecan link the different sections of text to their correspondingillustrations. For example, the engine can link a section of text withits related illustration to be displayed in the same screen. Theconsolidated layout information for text and illustration can then befed into the screen rendering engine. In one embodiment, based on thefed information, the screen rendering engine can retrieve, for example,text from the text sub file and illustrations from the illustration subfile, and digitize the appropriately-assembled text and illustrationsfor display in the screen.

In one approach, to render an illustration, a screen layout moduleretrieves, for example, attributes, such as format and positioninformation, of the illustration from an illustration descriptor. Theattributes can include a file name and a file type. In FIG. 8b , theillustration descriptor includes the filename “grevious.swf”, whichindicates the location of the illustration as the file namedgrevious.swf. The filename also indicates a file type “swf”, typicallydenoting an Adobe Flash file. Then the screen layout module can runAdobe Flash Player for the illustration. In this example, theillustration descriptor may include attributes for the Adobe FlashPlayer software, such as allowing for fullscreen, allowing to pause, andothers. The screen layout module can invoke a Flash player module, andpass the filename “grevious.swf” to the Flash player module.

In one embodiment, a screen rendering engine can perform the function ofscaling. For example, an illustration is a picture with a resolution of400×400 pixels. The screen layout module sets the dimension for theillustration to be a square of 100×100 pixels. This can be due to spacein the screen. Then the screen rendering engine can scale the picture tofit into the 100×100 pixel square area.

FIG. 9 shows one embodiment where information from an illustrationdescriptor is also fed into a screen rendering engine. In anotherembodiment, such information is fed into a screen layout engine, but notdirectly into the screen rendering engine.

The screen layout process can be implemented at different times. In oneembodiment, when a reader gets an e-book for a reading device, the site(which can be a web site) providing the e-book can access the readingdevice to find out attributes regarding the reading device, such as itsscreen size. Based on such information, the site can perform thecalculation for the screen layout of the e-book, or can perform thecalculation for the screen layout of at least a portion of the e-book.For example, the site can assemble at least a portion of the contents inthe e-book to be displayed in the reading device, and transmit suchassembled information to the reading device. In another embodiment, as areader is reading an earlier part of the e-book, the screen layout of alater part of the e-book is assembled. The calculation can be performedone portion at a time. For example, as the reader starts reading onechapter, the layout for the next chapter is calculated. In anotherexample, while the reader is reading one page, the layout for the nextone or more pages is calculated, or their contents are assembled. Inanother approach, such calculation can be performed by the readingdevice.

There can be different approaches to implement a screen layout module tolayout contents in a page of an e-book for a reading device. In oneembodiment to layout the page that includes only text, first the moduledetermines the size of the screen of a display of the correspondingreading device, such as 800*600 for landscape; and 600*800 for portrait.In one example, a screen can be separated into sub screens, such as asub screen of 800*600 and another of 800*120. Then the module assumesthe font size for text, such as 14 points. Both the screen size and fontsizes can be configured to other values. The text in the page to bedisplayed is retrieved, which can be done a section at a time. Thescreen layout module can retrieve such information from the text subfile of the e-book. In another approach, other attributes regarding thetext or a section of the text, such as location information, are alsoretrieved, such as from the text sub file. And the module can calculatethe logical position of the section, and determine if the current screenhas sufficient space to display the section. If the calculated logicalposition of the section can be displayed within the current screen, themodule would set to display the section of text within the screen. Ifnot, the section would be moved to the next screen.

In another embodiment, the page only includes an illustration, which canbe an animation. The screen layout module can retrieve, for example,location information of the illustration from an illustration descriptorin an illustration sub file of the e-book. There can be multiplesections in the page with illustrations, and an illustration can have anillustration id (e.g. “ani-4” in the example shown in FIG. 8b ). Themodule can calculate the logical position of the illustration (e.g. forthe animation “ani-4” in the example shown in FIG. 8b ). If thecalculated logical positions can be within the current screen, themodule would set to display the illustration within the screen.

In one embodiment, a page includes both text and illustration. In thisexample, a text sub file includes information to specify the layout ofthe illustration. In another example, an illustration sub file includesinformation to specify the layout of the text, and such information canbe in an illustration descriptor in the illustration sub file.

In yet another embodiment, whether an illustration is displayed isdetermined by the reader. There can be a symbol or icon indicating thepresence of an illustration at a certain area or section on a page. Ifthe reader clicks the symbol or icon, the corresponding illustrationwould be displayed, which can be in a pop-up manner. Similarly, if thereader is watching illustrations, in one embodiment, there can be asymbol or icon indicating the presence of text. If the reader clicks thesymbol or icon, the corresponding text would be displayed.

In one embodiment, there can be multiple sections in a page, with somehaving text and illustrations and some without illustrations. Oneapproach to layout a screen in such the page is to fix the positions ofthe text or illustrations in the screen. The following serves as anexample where the positions of illustrations are fixed. The screenlayout module pre-determines the positions of illustrations in thelayout of the screen. The module can reserve a side bar, such as an area(e.g. 25% of the screen size) on the left side or the right side of thescreen, for illustrations, when the screen is in landscape mode. Or,when the screen is in portrait mode, the module can reserve an area(e.g. 20% of the screen size) on the bottom of the screen forillustrations. In another approach, the module can reserve a number ofboxes, such as square boxes of various sizes, on the screen forillustrations. Then, in determining if an illustration can be shown, themodule can retrieve the illustration's dimensions from an illustrationdescriptor, and calculate the remaining available space in theillustration boxes/areas to see if the illustration can fit. In oneapproach, the module can adjust the size of the illustration forfitting. For example, if the available/remaining space is 100 by 100pixels and the illustration is 120 by 120 pixels, the module can reducethe size of the illustration by about 50% to fit the illustration intothe available area. In one embodiment, the module is allowed to scalethe illustration by not more than a pre-determined percentage. Iffitting requires more than the percentage, the illustration with thecorresponding text, if any, could be moved to the next screen.

FIG. 10 shows one embodiment to layout contents in a screen for a pagethat includes text and illustrations. In this example, the screen layoutmodule reserves 40% of the screen size on the bottom of the screen forillustrations. First the module determines the size of the screen andthe pre-determined font size for text. Then the module retrieves a pieceof text (e.g. for the next section to be displayed) and at least oneattribute regarding the text, and calculates the logical positions forthe piece of text. If the calculated logical positions of the piece oftext are not within the 60% of the current screen (or within theremaining empty portion of the 60% of the screen), the module would movethe piece of text, with all subsequent sections of the page, to the nextscreen (or the module can change the font size of the text, or performother operations). If the piece of text can be within the 60% of thecurrent screen, the module would determine if there is a correspondingillustration to be displayed with the piece of text. For example, ifthere is such an illustration, there can be an illustration id linked tothe piece of text. If there is no corresponding illustration, the pieceof text can be displayed. If there is such an illustration, the modulecan retrieve at least one attribute regarding the correspondingillustration for display. Such information can be in an illustrationdescriptor in an illustration sub file. Based on the retrievedinformation, the module can calculate the logical positions of theillustration to see if the illustration can be within the 40% of thescreen (or within the remaining empty portion of the 40% of the screen).If the illustration can be displayed, the module would retrieve theillustration, and display the illustration with the corresponding pieceof text in the current screen. If the illustration cannot, the modulewould move the illustration with the corresponding piece of text to thenext screen.

In one embodiment, the content in a page extends beyond one screen. Thescreen layout module can assemble the content in the page in consecutivescreens, but maintain the pagination of the e-book. In other words, theconsecutive screens can have the same page number. In one approach, thescreen layout module would set to display the text with thecorresponding illustration, if any, that can be shown within one screento be shown in that screen. The areas that extend beyond the screenwould be moved to the next screen. The module can redo the calculationfor those areas, while maintaining the pagination.

In one embodiment, when the last part of a page is showed on a screenand there is still space in the screen, the screen does not fill thespace with the beginning part of the next page. In other words,additional information from the next page is not introduced into thecurrent screen for this embodiment.

In one embodiment, the screen layout module does not provide scrollingfunction to show only a portion of an illustration. Instead, the moduleprovides a discreet jump function where an entire illustration is movedfrom one screen to the next screen, without allowing an illustration tobe partially shown on a screen.

FIG. 11 shows one embodiment of an annotation descriptor based on XMLand can be in an auxiliary sub file. This example shows a textannotation. This example differs from a text sub file with embeddedillustration id to show the existence of illustration linked to a pieceof text. One reason for the difference is because an annotation can beprovided subsequent to the publishing of an e-book. In this example,information in an auxiliary sub file regarding different annotations candetermine if there is an annotation linked to a piece of text. Thisdetermination can be done, for example, by examining or searching theauxiliary sub file to identify one or more annotations for, such as, aspecific section of text or illustration of the e-book. The annotationfor a corresponding section can then be identified, and subsequentlyshown on the screen. Also, for the annotation, its different attributes,such as its dimensions (computed, for example, by font size, text,images, etc.), can be determined. Then the annotation can beappropriately linked and displayed with the appropriate section. In oneexample, there can be reserved areas on a screen to display theannotation; the annotation can be shown in a pop up window if activated;or other approaches described in the specification can be used. If thecurrent screen does not have enough space to display the annotation, theannotation with the corresponding section can be moved to the nextscreen.

FIG. 12 shows one embodiment of a collection of annotations based onXML. The collection can serve, for example, as a study guide or ane-book review, and can be in an annotation descriptor. The collectioncan be from one or more other readers. In the example shown in FIG. 12,the collection of annotations is from one author, Old-Grievous, who islinked to the collection of annotations. In this example, theannotations can serve as a book review for an e-book.

In one embodiment, a reading device includes a reader analyzer todetermine, for example, at least an attribute of a reader. The analyzercan make such determination based on information collected by a readermonitor. The monitor can store the monitored reader's reading behaviorin an auxiliary sub file. In a different embodiment, the reader monitorcan include a reader analyzer, or some functions of the reader analyzercan be performed by a reader monitor. FIG. 7 shows a number of readingbehaviors identified by a reader monitor according to differentembodiments. FIG. 13 shows a number of attributes regarding a readeranalyzer, which can use information captured by the reader monitor foranalysis.

In one embodiment, a reader analyzer analyzes a reading behavior of areader. For example, the analyzer can determine a typical reading paceof the reader in reading an e-book. This can be done based on averagingthe reader's reading speed over a number of pages/chapters of thee-book. In another embodiment, the analyzer can be configured toidentify the reader's interest (or degree of interest) and/or progressin an e-book and/or a section of an e-book. This can be based onanalyzing a reading behavior of the reader. For example, a section/pagethat the reader has read more times than other section/page indicatesthat the reader may be more interested in that section/page. Asection/page that the reader has spent a lot of time on may alsoindicate the reader's higher interest in that section/page. Anotherreading behavior the analyzer can analyze is based on the reader'shighlight and/or the reader's notes. A highlighted section/page is asection/page that the reader is more interested in. A section/page thereader has annotated also can indicate the reader being more interestedin that section/page. Another reading behavior an analyzer can analyzeis based on whether the reader has clicked on a hypertext-link (or ahighlighted area) at a section/page. The reader can be considered moreinterested in the area the reader has clicked. One embodiment canmonitor which highlighted words have been clicked. One approach assumesthat the reader is more interested in subject matter(s) under clickedwords.

Yet another reading behavior the analyzer can analyze is based on thereader's images. One or more of the monitored behaviors can be combinedin the analysis. For example, no or very little page movement for a longduration (which can be relative to the reader's typically reading pacein reading the e-book) conveys very different meanings regarding thereader's interest in that page if the reader is present in front of thereading device, or absent from the reading device. In one approach, indetermining a reader's interest level, a reader analyzer determineswhether the reader is in front of the reading device.

In one embodiment, a reader analyzer can analyze to determine a reader'sconcentration level in reading an e-book, or section(s)/page(s) of thee-book, based on the reader's one or more images, and/or the reader'sreading behavior, which can depend on information in the one or moreimages. Different embodiments regarding a reader's concentration levelcan be found in U.S. Pat. No. 5,944,530, entitled “Learning method andsystem that consider a student's concentration level,” which is herebyincorporated by reference.

In one embodiment, a reader analyzer can also analyze to determine areader's understanding in an e-book, or section(s)/page(s) of thee-book. One approach to determine a reader's understanding level isagain based on analyzing one or more reading behavior of the reader. Forexample, a page includes highlighted words with hypertext linkages to adictionary. One embodiment monitors which highlighted words have been orhave not been clicked or selected on a page. In one approach, if thereader has selected a specific word to find out its meaning from thedictionary, the reader is considered to have understood the meaning ofthe word.

In another approach, an e-book is generally suitable for 5th gradereading level, but with certain words at different levels, such as at6th grade level or higher. Based on the words selected by a reader tocheck for meaning, in one embodiment, the reader's vocabulary level canbe determined.

As another example, an e-book is a user manual, such as of a dishwasher.The manual can highlight terms and/or functions specific for theproduct/service, which in this case can be a dishwasher. Eachhighlighted word can be hypertext linked, such as a highlighted functioncan be hypertext linked to an animated video showing how the functioncan be used. If a reader has selected that highlighted word and haswatched the linked video, the reader is considered to have understoodhow to operate the function.

As yet another example, an e-book is a textbook, such as a mathematictextbook. Some highlighted words can be mathematical terms and arelinked to their corresponding definitions in the e-book. Another termcan be further linked to a list of related theorems, lemmas and/orcorollaries. If a reader has selected a highlighted mathematical termover a predetermined number of times, such as over 20 times, a readeranalyzer will assume that the reader does not understand the term. Theanalyzer can also keep track of the time when the reader has selected aterm and make decisions based on the time/durations/gaps when the readerre-selects a term. For example, if the reader has selected the term“Fibonacci Series” a third time within a predetermined time frame, whichis considered short, such as a week, an analyzer can assume that thereader is very interested in the term. In addition to the definition ofthe term, additional related materials can be presented to the reader,such as an animation generating a golden spiral, or counting spiralingflorets of a Sunflower head.

Another approach to determine understanding level depends on having areader monitor ask a reader a question. This includes sending the readera quiz or a number of questions. In one example, asking a question canbe asking the reader to perform an activity, such as on an illustration.The results can determine the reader's understanding in certainsection(s)/page(s) of an e-book. As an example, an annotation of asection/page from another reader is showed to the reader and then thereader monitor asks the reader for agreement or disagreement with theannotation shown. Based on the response, the reader analyzer can have anindication on the reader's understanding level.

In analyzing a reader's behavior to determine, for example, the reader'sinterest, progress and/or understanding regarding an e-book, or otherattributes, a reader analyzer can make the determination relative to thereader's past behavior, or relative to other readers. For example, thereader's typical reading speed is five minutes a page (which can berelative to a specific e-book). If the reader is much slower than thisrate (for example, for the same e-book) at a certain page, the readermight be more interested in that page, or might need help in theinformation in that page. In another embodiment, the analyzer cancompare an attribute of the reader with an average measurement of theattribute of many other readers. For example, most readers spend 5minutes on a page for an e-book, but the current reader has been at thatpage for a long time (e.g. in front of a camera of the e-book and hasnot moved from the page for 20 minutes). This may imply that thereader's interest in that page is higher than typical, or the reader mayneed help in that area.

One embodiment identifies the subject matter of a section/page or anarea the reader is reading. Based on the subject matter identified, areader analyzer can determine an attribute regarding the reader. In oneapproach, each section/page/illustration can be characterized orrepresented by one or more key words, which can be meta tags. If areader is interested or is considered interested in an area, theircorresponding key words show the subject matter the reader is interestedin. In another approach, the analyzer identifies the subject matter ofan area/section of an e-book based on natural-language processing thecorresponding section/page. For example, the analyzer identifies thesubject matter of the area based on such as the grammars, semantics,keywords, terms, synonyms and/or phrases of the correspondingsection/page.

In one embodiment, a reader responder can be configured to react orrespond to a reader based on at least one attribute of the reader. Theattribute can be the reader's interest, progress and/or understanding insection(s) in an e-book, which can be determined by a reader analyzer.In another example, the response can be based on an output from a readermonitor. FIG. 14 shows a number of embodiments regarding attributes of areader responder.

In one embodiment, a reader responder can ask if the reader wantsadditional materials related to a section/page or an area of an e-book.This section/page may be what the reader is interested in, where thereader has progressed to, where the reader may need more assistance,and/or for other reasons.

The additional materials can be from different sources. In oneembodiment, the additional materials can be in the reading device thatthe reader is reading from, but may previously be locked from thereader, or previously not presented to the reader. The responder canunlock or present them to the reader to view under certain predeterminedcondition(s). In one example, the reader may need to pay to unlock or tosee the materials. In another example, the reader may have to belong toa certain organization/club, which can be an e-book club, in order tohave the privilege to view such additional materials.

In another embodiment, the additional materials are remote from thereading device the reader is using to read. The device can have acatalog of titles/subjects regarding additional materials, but not thecorresponding more detailed content. After the reader has identifiedcertain additional materials (such as based on the titles), the devicecan, wirelessly or by wire, link to an e-book club or a website (orother locations) and retrieve the certain additional materials into thereading device. In one approach, the certain additional materials can beadded, such as via overlaying, into an auxiliary sub file of the e-book.In another embodiment, the catalog of titles/subjects can also be at thewebsite (or the other locations), which can be accessible by the reader.

In yet another embodiment, additional information/materials can residein a memory device or storage medium of the creator/owner of theadditional materials. Such materials can be accessed via a peer-to-peerprotocol between the reader's reading device and the creator's device,which, in one embodiment, also can be a reading device. In anotherexample, the memory device or storage medium can be wirelessly accessedby the reading device of the reader.

There can be other types of reactions by a responder. In one embodiment,the responder can provide review regarding certain materials in ane-book for the reader. For example, in an area that the reader needsmore assistance, a responder can retrieve additional materials to helpthe reader. The additional materials can be related to, but do not haveto be the same as, the certain materials to be reviewed. For example,the certain materials include an animation; to review, a differentanimation can be played for the reader. Different embodiments regardingreview can be found in U.S. Pat. No. 5,863,208, entitled “Learningsystem and method based on review,” which is hereby incorporated byreference.

In one embodiment, a responder can provide rewards for a reader when thereader reaches a pre-determined milestone. For example, the reader hasfinished reading 50% of the e-book. The responder can ask if the readerwould like to have a reward. In another approach, a reader gets a rewardbased on the reader's understanding in the materials read. For example,a monitor can quiz the reader, and a reader analyzer can analyze theanswers to determine if the reader has understood the materials. Thereward can be tailored to what the reader is interested in. Differentembodiments regarding rewards can be found in U.S. Pat. No. 5,743,746,entitled “Reward enriched learning system and method,” which is herebyincorporated by reference.

In one embodiment, a responder can restrict a reader from certainactivity under a pre-determined condition. For example, if the readerhas not finished reading 10% of the e-book by a certain predeterminedamount of time, the responder can restrict the reader from serving theweb with the reading device. Different embodiments regarding restrictioncan be found in U.S. Pat. No. 5,743,743, entitled “Learning method andsystem that restricts entertainment,” which is hereby incorporated byreference.

In one embodiment, a responder can guide to a reader, products/servicesfor the reader to purchase and/or acquire. The product/services can bebased on the reader's interest. In one embodiment, the product is not ane-book and is not an expression. The product does not have to be aphysical product. For example, a product can be an access to a videogame for a duration of time, or a virtual product for an avatar, whichcan be related to the reader.

In another example, a product can be a reward for a reader, such as abookmark. Let say after a reader has read the first two chapters of ane-book, a responder can allow the reader to select a bookmark of thereader's choice, and the bookmark can be one of the characters, such asYoda, Darth Vader, Obiwan and Anakin. After finishing the first threechapters, the reader can select a matching lightsaber to go with thepreviously selected bookmark.

In yet another example, a product can be recommended to a reader to helpthe reader understand certain materials. Let say an e-book is aboutusing a piece of software, such as Microsoft Office 2010. A recommendedproduct can be a video showing in details how to include an Adobe Flashvideo into Microsoft Powerpoint 2010. Let say an e-book is on chemistryfor kids. A recommended product can be a crystal growing kit to help areader understand the concept of super-saturation.

In a further example, a product can enhance the reader's experience inreading an e-book. Let say a reader is reading about the fight betweenGrievous and Obiwan. A recommended service is a piece of epic music fromStar Wars Episode III, which can be played while the reader is readingthe fight. This product can be highly entertaining to the reader andenhances the reader's reading experience. Let say another product is acollection of high resolution pictures of Death Star and various TIEfighters. A user may find using such pictures as screen saverentertaining, and the pictures can enhance the reader's readingexperience in reading a Star Wars e-book. Such pictures on a readingdevice can even draw the reader back to reading the e-book when thereader is in another activity. Another way to enhance a reader's readingexperience can be to encourage a reader to share her reading experience.Let say the e-book is about gardening. An animated interactiveillustration of the e-book is to have the reader grow an orchid. Aftercompleting the illustration exercise of growing an Oncidium, a respondercan recommend the reader to email a picture of the grown Oncidium to afriend.

In one embodiment, before a reader can read an e-book via a readingdevice, the device can request the reader to enter an identification ora password. If the identification matches an identification stored inthe device, the reader can have access via the reading device to thee-book. In another embodiment, if there are no inputs to a readingdevice for a predetermined amount of time, the reading device may askthe reader to re-enter the identification. In one embodiment, theidentification is with respect to an e-book. A different e-book may needa different identification. Though the reader can use the readingdevice, the reader cannot read the e-book if the entered identificationdoes not match an identification stored for the e-book.

A number of embodiments have been described regarding an e-book and ane-book reading device. In one embodiment, at least one functionality ora portion of a component of an e-book reading device is in an e-book. Insuch an embodiment, the e-book reading device can lack certainfunctionalities to read an e-book (or to enjoy all the functionalfeatures provided by the e-book), and in one embodiment, suchfunctionalities can be in a logic sub file of the e-book. Instructionsin the logic sub file of the e-book complement or augment instructionsin certain functionalities of the reading device. As an example, atleast a portion of a reader analyzer is in the logic sub file of ane-book. As yet another example, at least a portion of a reader analyzer,at least a portion of a reader responder and at least a portion of ascreen layout engine are in a logic sub file of an e-book.

In one embodiment, instead of all the functionalities of a readeranalyzer and a reader responder residing in a reading device, a portionof the reader analyzer and a portion of a reader responder are in alogic sub file of an e-book. As an example, the e-book is a childrenchapter book on Star Wars with at least four versions, corresponding tofour age groups, namely 3-5 (pre-school and kindergarten), 6-8 (grades1-3), 9-11 (grades 4-6) and 12-13 (middle school). In a screen, for agegroup from 3 to 5, an image of Yoda and R2D2 are used as background,with the corresponding text having a very large font size. For age groupfrom 6 to 8, an image of General Grievous and Darth Vader are used, withthe corresponding text having a large font size. For age group from 9 to11, an image of a sword fight between Darth Vader and Obiwan is used,with the corresponding text in a regular font size. And for age groupfrom 12 to 13, images of various space vehicles are used, again with thecorresponding text in a regular font size. These older kids can clickany of the space vehicles to bring in additional materials regarding thevehicles for further reading. The version to be shown on a screen of adisplay of an e-book reading device depends on the age of the childreading the e-book.

In the above example, a reader monitor (or a reader analyzer) in areading device can first determine the child's age. Based on thedetermination, the reader analyzer in the device concludes that sincethe child is in the age group of 3-13, the children chapter book isappropriate for the child. However, the reading device lacks thefunctionalities of categorizing the child's age into one of the four subage groups (which can be performed by a reader analyzer), and ofselecting the version to be presented to the child based on the category(which can be performed by a reader responder). At least initially, inone embodiment, these functionalities can reside in the e-book. Based onthe age of the child, out of the four age groups, a reader analyzer (orthe portion of the reader analyzer), which can be in a logic sub file ofthe e-book, can categorize the child into one of the age groups or oneof the pre-determined categories. Then a reader responder (or theportion of the reader responder), which can be in a logic sub file ofthe e-book, can select the specific version of the e-book (or canretrieve at least a different section of the e-book) to be presented tothe child based on the categorization. To implement such approaches, inone embodiment, when the e-book is downloaded to a reader, the e-bookqueries the reading device as to whether the reading device has thefunctionalities to select version to be presented to a reader based onthe age of the reader. If the device response is negative, the e-bookwould download such functionality into the e-book reading device tosupport such functionalities.

As another example, terms in an e-book can have different definitions ormeanings depending on the age of the reader. An e-book includes adictionary (a reading device can include another dictionary). In oneapproach, a reader can select a word (which can be a highlighted word)in the e-book, and one of the dictionaries, such as the dictionary inthe e-book, is selected to provide the meaning of the word to thereader. Depending on what age group the reader is categorized, thereader can get a different meaning of at least one word. Using the word“force” as an example, for a young reader, its standard dictionarydefinition may not be as appropriate or interesting as an animationshowing Yoda using the force to stop a large concrete column fromcollapsing onto Anakin and Obiwan. However, an older reader can get atext definition of the word “force” as “The Force is what gives a Jedihis power. It's an energy field created by all living things. Itsurrounds us, penetrates us, and binds the galaxy together. It couldonly be harnessed by those with high count of microscopic life formscalled midi-chlorians. The highest known midi-chlorian count belonged tothe Anakin with over 20,000 per cell.” In the text, both the terms Jediand midi-chlorian can also be highlighted for additional explanation, ifselected by a reader.

In one embodiment of the dictionary example, an e-book can include areader analyzer and a reader responder with the functionality ofselecting a dictionary or a meaning of a word based on the age of achild. In another embodiment, an e-book reading device lacks suchfunctionalities. Then in one approach, when the e-book is downloaded tobe read at the e-book reading device, such functionalities can bedownloaded as well. In yet another embodiment, the e-book includes adictionary that provides meaning or explanation to words or terms basedon the age of a child. And when the e-book is downloaded to a readingdevice, the dictionary in the e-book can be downloaded also.

In one embodiment, a reading device includes a reader analyzer and oneor more general reading rules, while a logic sub file of an e-bookincludes another reader analyzer and one or more other reading rules,which can be content specific. FIG. 15 shows an example of such a logicsub file in an e-book. In this example, an e-book logic sub fileidentifies different collections of words or terms in the e-book fordifferent purposes. For example, the e-book is about Star Wars. Onecollection (first collection) includes “Darth Vader”, “Grievous”,“Sith”, and “Emperor”; and another collection (second collection)includes “Obiwan”, “Yoda”, “Force”, and “Jedi”. In this example, bothcollections of words can be hyperlinked in a text sub file of the e-bookfor additional information. A reader monitor, which can be in thereading device, monitors the words a reader has selected. The e-booklogic sub file includes a reader analyzer, which can be coupled to areader monitor of the reading device. Based on the words selected by thereader, the analyzer from the e-book generates two counts. TheDarkSideCount indicates the number of words selected that belong to thefirst collection, and the LightSideCount indicates the number of wordsselected that belongs to the second collection. From the two counts, thereader analyzer from the e-book determines if the reader is moreinterested in the light or the evil side of the story. A higher count inone indicates the reader's higher interest in that side of the story.Based on such determination, a reader responder can respond accordingly.For example, the responder can introduce different materials for thereader based on the reader's interest.

In yet another embodiment, one or more of (or at least a portion of oneor more of) a screen layout module, a note selector, a note editor, areader monitor, a reader analyzer, a reader responder and a controllerreside in an e-book, such as in a logic sub file of the e-book. In oneembodiment, when the e-book is being loaded into a reading device, thee-book queries the reading device if the device has one or more of theabove components or different features in the different components. Inone approach, any components or part of a component or functionalitiesthe device does not have, the e-book would load them into the readingdevice. Reader permission may be required for the down loading. In yetanother embodiment, an e-book is downloaded from a web site, an Internetportal, an e-book club or other locations (wired or wirelessly) to areading device. The different components mentioned above can be part ofthe e-book and can reside, with the e-book, in the website, the Internetportal, the book club or other locations, which can query the device ifthe device has the different components. Any components or part of acomponent the device does not have, such component(s) can be downloadedinto the reading device.

In yet another embodiment, one or more functionalities to read an e-bookreside in the e-book. As the e-book is read, the one or morefunctionalities can be sent or downloaded to the reading device used toread the e-book. For example, a chapter has different versions forchildren of different age group. When the chapter is about to be read(such as when the immediate previous chapter is being read orretrieved), the e-book downloads or sends the functionality of selectingversions based on age group to the reading device. In one embodiment,without the one or more functionalities, the e-book can still be read,except that the reading experience can be enhanced with the one or morefunctionalities.

One embodiment includes an e-book with a text sub file and a logic subfile. The text sub file can be configured to store at least a piece oftext, together with at least an attribute regarding the text stored. Thelogic sub file can be configured to include a set of instructions toinstruct an e-book reading device on a functionality regarding readingthe e-book at the device. The functionality can include analyzing anattribute of the reader regarding reading the e-book, and providing aresponse to the reader via the device in view of the attribute analyzed.In another embodiment, the e-book further includes an illustration subfile to store at least an illustration, together with at least anattribute regarding the illustration. Either the text sub file or theillustration sub file includes a piece of position information linkingthe illustration to an area at the at least one piece of text.

The following shows examples of e-books that include one or morefunctionalities to read the book.

In one approach, the e-book is a textbook to learn a subject, and thee-book is linked to another lecture-note e-book that includesassignments regarding the subject. A reader analyzer in the lecture-notee-book can analyze the progress of the reader. Based on the analysis onthe reader's performance level (and may also include instructionalneeds), a reader responder in the lecture-note e-book can guidedifferent assignments to the reader. Both of the reader analyzer andreader responder can be in a logic sub-file of the lecture-note e-book.

Assume Professor Hoperoft is teaching a class in Automata Theory basedon text with a number of chapters in a text e-book, and a lecture-notee-book. Each chapter in the text e-book has its corresponding lecturenote in the lecture-note e-book. Materials in the text e-book includeillustrations, which are animations of automata such as finite statemachines and pushdown automata. The lecture-note e-book includeshighlighted areas/terms hyperlinked to areas in the text e-book. Forexample, terms in automata theory in the lecture-note e-book are linkedto their definitions in the text e-book, as well as related theorems,lemmas, corollaries in the text e-book. There are assignments in thelecture-note e-book. And, there are illustrations in the assignments.The lecture-note e-book includes a reader analyzer and a readerresponder.

With chapter 2 as an example, Professor Hoperoft puts together 5animation illustrations within the text e-book and 10 assignmentillustrations in the lecture-note e-book. Each student should go throughall animation illustrations within the student's text e-book, and 3 ofthe ten assignments in the student's lecture-note e-book after finishingthe text e-book. Assignments in the lecture-note e-book have highlightedareas/terms linked to the text e-book for term definition and varioustheorems. Each of the 3 assignments includes a hint. If a student has toselect all three hints to finish the 3 assignments, Professor Hoperoftbelieves the student's understanding level is weak and the student willbe asked to work on 6 of the remaining 7 assignments to help the studentget more practices. The last remaining assignment is a more difficultassignment and is reserved for students who finish all 3 of the requiredassignments without selecting any hint. The reader analyzer in thelecture-note e-book monitors its corresponding student, and determineshow many hints the student has selected. Based on the number of hintsthe student has selected, the reader responder in the lecture-notee-book will bring in the corresponding number of assignment(s) for thestudent. Furthermore, the reader responder is programmed to sendelectronic messages to Professor Hoperoft regarding the student'sperformance, such as informing the professor of the student working onthe more difficult assignment (or the well performing student) orworking on the 6 additional assignments (or the poor performingstudent).

For Chapter 2, Professor Hoperoft schedules a week to finish the textportion of that chapter in the text e-book. The amount of time expectedto complete different chapters can be different and such expected timeframes are stored in the analyzer of the lecture-note e-book. The readeranalyzer monitors the time duration to finish a chapter (both the textand the assignments). For example, the reader analyzer monitors the timebetween starting the first page to ending the last page of the text ofChapter 2. If the duration is larger than the expected time of one weekto finish Chapter 2, the 6 additional assignments will be shown to thestudent. If a student needs significantly shorter amount of time thanthe expected time (say 3 days) to finish Chapter 2, the more difficultassignment (assignment 7) will be shown to the student. In both cases,the responder in the lecture-note e-book provides periodic studentprogress reports to Professor Hoperoft via electronic messages.

In the above examples, the lecture-note e-book and the text e-book aretwo different e-books. There are other ways to implement a lecture-notee-book. One approach is to integrate the lecture-note e-book with, ormake the lecture-note e-book to be, a portion of the text e-book.Professor Hoperoft can make the lecture-note e-book to be an addendumadded to the text e-book. Professor Hoperoft can also make thelecture-note e-book to be a revision to the text e-book.

In another approach, e-books with functionalities to read them can bereference e-books, such as a city travel guide; a restaurant guide; acook-book; a do-it-yourself reference book for gardening, constructing(a building, a driveway pavements etc.), landscaping, home electronics,or home computers; a product user-manual; a photography handbook; orother types of reference books.

Depending in the specialty/industry, the reference e-book includesdifferent specific terms or jargons. A city travel guide could include anumber of city names, district names, restaurant names, types ofcuisines, cultural terms, and/or city-specific or locale-specific words.A cook-book could include a number of ingredients, cooking techniques,names of dishes, names of chefs and authorities, and/or utensil namesfor different types of food. A gardening book could include a number ofbotany terminologies, plant names, planting techniques and/orequipments. A do-it-yourself book could include a number of brands forparts and components, specific terms to do certain tasks, mechanical oroperation procedures, and/or special tools. In any event, a referencee-book highlights these terms and hyperlinks texts/illustrations (asannotations) to further explain them. Such illustrations are pictures,animations or videos.

Assume a reference e-book is a city travel guide with informationregarding a restaurant named Chef Tom in San Francisco. After a readerhas visited the restaurant, the reader can create an annotation and addthe annotation to the restaurant in her city travel guide. Theannotation can be a picture or video and a piece of descriptive text. Inone approach, the reader can allow her annotations to be read by others,such as from an Internet portal. The e-book allows the reader to markher annotation(s) as sharable. Then other readers reading their own citytravel guide would be able to access her annotation. For example, whileanother reader is reading his city travel guide and comes across ChefTom, her annotation can be read by this another reader.

An annotation can be on another's annotation. For example, a readerretrieves an annotation created by another reader about the restaurantChef Tom in the e-book. The reader decides to add a new annotation tothe retrieved annotation. If the reader designates the new annotation assharable, other readers would be able to access both the originalannotation with the new annotation.

Richard, a reader of his city travel guide (his e-book), can rankdifferent points of interest. The guide includes a piece of programminglogic in a logic sub-file that allows Richard to give a ranking to apoint of interest, such as a restaurant, using 1-star to 5-star. Thisranking is an annotation to the restaurant. Again in one approach, theannotation is sharable (or can be made sharable by Richard) and can beread by other readers. This can be done via an Internet portal, whichcan be the portal Richard gets the guide. Other readers can also entertheir ranking to the restaurant to their own e-books. In one embodiment,another piece of programming logic in Richard's guide can prompt Richardto enter a ranking to the restaurant. The logic sub-file can alsoinclude a piece of programming logic that averages the sharable rankingsentered by all readers to create a new annotation for the restaurant.Then Richard reading his guide can see the average ranking, togetherwith individual rankings from different readers about the restaurant.

Assume another reference e-book is a cook book with information ondifferent recipes and the book is read via an Internet portal, awebsite, or an e-book-reading social networking web entity. In oneapproach, different readers can read their e-books, and their e-books'content can change as readers add annotations. For example, a readerfollows a recipe to create a dish, and he videotapes the cookingprocess. The reader's e-book allows him to attach the video clip as anannotation to the recipe in the e-book. The reader further adds writtencomments to the annotation (or as an annotation to the video clipannotation). The comments are on his experience in making the dish, suchas substituting a different ingredient, or using a different brand,leading to a longer cooking time, but a tastier dish. For another dish,the reader creates an annotation with a picture of a similar dish heorders in a restaurant. He further adds to the picture annotation on howthe presentation, aroma and taste of the restaurant dish differ from thedish made following the recipe in the e-book. And the reader makes eachof the above annotations sharable via the Internet portal. Note that ifthe reader is a renowned food critic, his annotation can be rankedhigher (e.g. shown up at the top of the list) than other annotationsregarding the dish. Such ranking can be done manually. For example, theInternet portal has editors who could adjust the positioning of theannotation. Another way is based on comments on/rankings of annotations.These comments/rankings can themselves be annotations. A highly rankedannotation moves up the list.

A number of embodiments have been discussed regarding an e-book beingread. In one embodiment, an e-book or some of the expressions in thee-book can be listened to through an e-book reading device that includesat least a speaker. In reading out an e-book, the e-book reading devicecan also announce the page number of the page that is being read.

A number of embodiments have been discussed regarding an e-book. In oneembodiment, an e-book can be licensed to a reading device. In oneexample, the license can be extended to a certain number of readingdevices that belong to the reader. In another embodiment, a readingdevice does not have to acquire the entire e-book, but can acquirecertain pages/sections of the e-book, or a portion at a time. In yetanother embodiment, the reader gets an e-book based on subscription orrental, and the subscription or rental can be based on time. Forexample, the reader can pay a fee through or to a website. Then areading device obtains the e-book from the website, and the reader canstart reading the e-book using the reading device. Once the subscriptionor rental is over, the reader would be stopped from reading the e-bookfrom the reading device.

Different types of e-books may be more applicable to be read bydifferent embodiments. For example, an e-book can be a cookbook withvideo, a travel book with video, a product menu with video andanimation, and textbooks with animation (such as school textbooks),children books, journals, magazines, newspaper and other types ofreading materials. In another embodiment, an e-book includesconsolidations of reading materials, such as one or more other e-books,journals, newspapers, blogs and/or other materials.

FIGS. 16A-B show examples to implement one or more embodiments,preferably in software and hardware. In FIG. 16A, the embodimentincludes a server system and a number of client computers. The serversystem can include a number of server computers. A client computer canbe a personal computer (or computing device), and each client computercan communicate to the server system through a dedicated communicationlink, a computer network or a network interface.

In one example, a server system 652 includes more than one servercomputer. For example, a server computer includes a processing unit, amain memory and a network interface. And the server computer can alsoinclude or be connected to an external storage unit. The networkinterface allows the server computer to communicate with differentexternal devices, if any, such as other server computers, an externalstorage unit and/or a client computer. The external storage unit can be,for example, a harddisk drive, which can be a solid state drive such asFlash memory, a memory card, an USB drive, a floppy disk drive, and anoptical disk drive.

In one embodiment, multiple server computers can communicate with eachother via a network interface. For example, an e-book club (or a websiteproviding an e-book to a reading device) includes a server computer (ora server system) and another website on newspapers includes anotherserver computer (or another server system). These computers or systemscan wired or wirelessly coupled together to communicate with each other.

In one embodiment, a server computer includes a Web server or a softwareprogram that runs HTTP protocol or related Web protocols.

FIG. 16B shows one embodiment of a client computer. It typicallyincludes at least a bus 659 connecting a number of components, such as aprocessing unit 660, a main memory 662, an I/O controller 664, aperipheral controller 666, a graphics adapter 668, a circuit board 680and a network interface adapter 670. The I/O controller 664 can beconnected to components, such as a harddisk drive 672 (which can be asolid state drive such as Flash memory, a removable disk drive such as amemory card, or an USB drive), a floppy disk drive, a removable diskdrive, an optical disk drive, or a network attached storage device 674.The peripheral controller 666 can be connected to one or more peripheralcomponents, such as a keyboard 676, a pointing device 682, or a touchdevice 683. The graphics adapter 668 can be connected to a monitor 678.The circuit board 680 can be coupled to audio or video signals 681; andthe network interface adapter 670 can be connected to the network 656,which can be the Internet, an intranet, a wireless network, a mobilecellular network, the Web or other forms of networks. The processingunit 660 can be a microprocessor, a field programmable gate array, or anapplication specific chip. In another embodiment, a client computer 654is a thin-client, with less computation capability and memory capacitythan the server system, 652. In one embodiment, a client computer 654includes better computation capability than a server system 652, such asin graphics, video and user interface; in another example, to improvethe interaction with readers, a client computer 654 can have highermemory capacity than a server system 652.

In one embodiment, a client computer 654 can be configured as and/orfunction as an electronic reading device and as one or more of thefollowing: a handheld device, an e-book reader, a smartphone, anotebook, a mini-notebook, a smartbook, a net-top box, a set-top box, agame console, a portable handheld gaming device, a portable computingdevice, an in-car system, a GPS navigation system, a media telephone, amedia player such as an MP3 player or video player or blu-ray player, adigital signage, an electronic kiosk, a computing tablet, a mediacenter, an electronic newspaper or magazine reader, an electronic toy,an electronic dictionary, and an electronic phonebook.

Different elements/systems in different embodiments may be in differentcomponents. For example, a reading device may be in a client computer.In another embodiment, a reading device can be in a client computer, andsome of the e-book contents being read by the device can be in a storagemedium not in the client computer, but in a server system instead. Inyet another embodiment, a server system hosts an e-book club or awebsite providing e-books.

Different systems/components in different embodiments can be implementedin hardware or software or both. For example, a note selector and a noteeditor of a reading device can be written in software, or can be on acircuit, such as a field-programmable-gate-array, where the programembodying the note editor and note selector is burnt into a circuit. Asanother example, a processing unit 660 can implement the controller of areading device; and the same or a different processing unit canimplement a reader analyzer in a reading device.

The various embodiments, implementations and features of the inventionnoted above can be combined in various ways or used separately. Thoseskilled in the art will understand from the description that theinvention can be equally applied to or used in other various differentsettings with respect to various combinations, embodiments,implementations or features provided in the description herein.

The invention can be implemented in software, hardware or a combinationof hardware and software. A number of embodiments of the invention canalso be embodied as computer readable code on a computer readablemedium. The computer readable medium is any data storage device that canstore data which can thereafter be read by a computer system. Examplesof the computer readable medium include read-only memory, random-accessmemory, CD-ROMs, magnetic tape, optical data storage devices, andcarrier waves. The computer readable medium can also be distributed overnetwork-coupled computer systems so that the computer readable code isstored and executed in a distributed fashion.

Numerous specific details are set forth in order to provide a thoroughunderstanding of the present invention. However, it will become obviousto those skilled in the art that the invention may be practiced withoutthese specific details. The description and representation herein arethe common meanings used by those experienced or skilled in the art tomost effectively convey the substance of their work to others skilled inthe art. In other instances, well-known methods, procedures, components,and circuitry have not been described in detail to avoid unnecessarilyobscuring aspects of the present invention.

Also, in this specification, reference to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment can beincluded in at least one embodiment of the invention. The appearances ofthe phrase “in one embodiment” in various places in the specificationare not necessarily all referring to the same embodiment, nor areseparate or alternative embodiments mutually exclusive of otherembodiments. Further, the order of blocks in process flowcharts ordiagrams representing one or more embodiments of the invention do notinherently indicate any particular order nor imply any limitations inthe invention.

Other embodiments of the invention will be apparent to those skilled inthe art from a consideration of this specification or practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with the true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A non-transitory computer readable medium in amobile device for a user including at least materials stored therein,the materials comprising one or more text sub file including at least apiece of text, one or more illustration sub file including at least anillustration, and one or more logic sub file, with executable computerprogram code in at least one of the one or more text sub file and theone or more logic sub file, the mobile device including at least adisplay, the executable computer program code comprising: computerprogram code to layout at least a screen of a first size, with thescreen including at least the piece of text and the illustration atleast by keeping the piece of text with the illustration to be presentedby the mobile device, to fit into the screen of the first size, at leastbased on (a) the first size, (b) a font size of the piece of text, (c)numbers of pixels to present the illustration, (d) calculating at leasta position for the piece of text in the screen, and (e) calculating atleast a position for the illustration in the screen; and computerprogram code to layout at least a screen of a second size, with thescreen to include at least the piece of text and the illustration, atleast by keeping the piece of text with the illustration to be presentedby the mobile device, to fit into the screen of the second size, atleast based on (a) the screen of the second size including at least apreset area, and (b) changing the numbers of pixels to present theillustration to fit at least the illustration into the at least a presetarea in the screen of the second size, wherein the second size isdifferent from the first size.
 2. A non-transitory computer readablemedium as recited in claim 1, wherein the numbers of pixels to presentthe illustration includes a first number of pixels by a second number ofpixels, and wherein the changing the numbers of pixels includes reducingat least one of the first number and the second number.
 3. Anon-transitory computer readable medium as recited in claim 1, whereinthe numbers of pixels to present the illustration includes a firstnumber of pixels by a second number of pixels, and wherein the changingthe numbers of pixels includes scaling the illustration by changing atleast one of the first number and the second number.
 4. A non-transitorycomputer readable medium as recited in claim 3, wherein the scaling theillustration is not to reduce to below a predetermined percentage.
 5. Anon-transitory computer readable medium as recited in claim 3, whereinthe scaling the illustration is not to increase to beyond apredetermined percentage.
 6. A non-transitory computer readable mediumas recited in claim 1, wherein the first size includes a first number ofpixels by a second number of pixels, wherein the second size includes athird number of pixels by a fourth number of pixels, and wherein thesecond size is different from the first size such that at least thefirst number is different from the third number or the second number isdifferent from the fourth number.
 7. A non-transitory computer readablemedium as recited in claim 1, wherein for the screen of the first size,the screen is in one of landscape or portrait mode, wherein for thescreen of the second size, the screen is in the other one of landscapeor portrait mode, and wherein the screen of landscape mode includes afirst number of pixels by a second number of pixels, and the screen ofportrait mode includes the second number of pixels by the first numberof pixels.
 8. A non-transitory computer readable medium as recited inclaim 1, wherein the mobile device includes at least two displays, andwherein at least a portion of the screen of the first size is in one ofthe at least two displays, and at least a portion of the screen of thesecond size is in the other one of the at least two displays.
 9. Anon-transitory computer readable medium as recited in claim 1, whereinthe mobile device includes at least two displays, and wherein the screenof the first size is in one of the at least two displays, and the screenof the second size is in the other one of the at least two displays. 10.A non-transitory computer readable medium as recited in claim 1, whereinthe mobile apparatus includes at least two displays, and wherein atleast the screen of one of the first size and the second size isconfigured to occupy the at least two displays.
 11. A non-transitorycomputer readable medium as recited in claim 1, wherein the computerprogram code to layout the at least a screen of a second size is basedon the screen of the second size having another preset area for at leastthe piece of text.
 12. A non-transitory computer readable medium asrecited in claim 1, wherein the computer program code to layout the atleast a screen of a second size is based on the screen of the secondsize having another preset area to at least receive input from the user.13. A non-transitory computer readable medium as recited in claim 1,wherein the computer program code to layout the at least a screen of asecond size is based on the screen of the second size having anotherpreset for at least a piece of note, wherein the piece of note isrelated to at least an area of the materials stored in the computerreadable medium, and wherein the piece of note is configured to bewirelessly received by the mobile device.
 14. A non-transitory computerreadable medium as recited in claim 13, wherein the piece of noteincludes at least comments by another user regarding the at least anarea of the materials.
 15. A non-transitory computer readable medium asrecited in claim 1, wherein the computer readable medium comprises:computer program code to receive an input regarding a piece ofmaterials; and computer program code to layout the at least a screen ofthe second size to fit at least the illustration and the piece ofmaterials into the at least a preset area in the screen of the secondsize.
 16. A non-transitory computer readable medium as recited in claim15, wherein the computer readable medium comprises computer program codeto monitor the user to receive the input.
 17. A non-transitory computerreadable medium as recited in claim 15, wherein the computer readablemedium comprises computer program code to layout at least another screenof the second size to at least present the piece of materials at leastin view of not sufficient space to fit at least the illustration and thepiece of materials into the at least a preset area in the screen of thesecond size.
 18. A non-transitory computer readable medium as recited inclaim 1, wherein the screen of the second size includes at least anotherpreset area to present at least an area of the materials stored in thecomputer readable medium, and wherein the preset area and the anotherpreset area depend on the second size.
 19. A non-transitory computerreadable medium as recited in claim 1, wherein the materials stored inthe computer readable medium include an audio file in or linked to anarea in the materials, and wherein the computer readable mediumcomprises computer program code to play the audio file at least based onthe area in the materials being presented on at least the display.
 20. Anon-transitory computer readable medium as recited in claim 19, whereinthe area in the materials includes an illustration.
 21. A non-transitorycomputer readable medium as recited in claim 19, wherein the area in thematerials includes a piece of text.
 22. A non-transitory computerreadable medium as recited in claim 1, wherein the computer readablemedium comprises computer program code to render at least theillustration in the at least a screen of a first size according to afile extension of the illustration.
 23. A non-transitory computerreadable medium as recited in claim 22, wherein the file extensionincludes .swf.
 24. A non-transitory computer readable medium as recitedin claim 22, wherein the file extension includes .mp3.
 25. Anon-transitory computer readable medium as recited in claim 1, whereinthe illustration includes an animation, and wherein the computerreadable medium comprises computer program code to render the animationbased on a script regarding the animation.
 26. A non-transitory computerreadable medium as recited in claim 1, wherein the computer readablemedium comprises: computer program code to monitor an input of the userinto the screen of the second size; and computer program code to layoutat least a screen of the second size, with the screen to include atleast the input and an area in the materials to be presented by themobile device, to fit into the screen of the second size.